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1. Field of the Invention 

The present invention relates to the field of textual or non-verbal 
communication. More specifically, the present invention relates to methods and 
apparatuses associated with the employment of emoticons in textual or non-verbal 
communications, such as email or instant messaging, practiced on e.g. mobile 
communication devices, as in the case of wireless mobile telephones. 

2. Background Information 

Advances in integrated circuit, microprocessor, networking and 
telecommunication technologies have led to the development and wide spread 
adoption of networked computing devices and mobile communication devices. As a 
result, email, instant messaging, and wireless telephone calls have all become 
important vehicles of communication for a large segment of the population, if not the 
preferred forms of communication for these users. 

It is known that for many users, their email and instant messaging 
communications (also referred to as textual or non-verbal communications) often 
involve the use of emoticons, such as the "smiling face" or the "sad face". However, 
few email or instant messaging applications offer any assistance to a user to enter 
and use emoticons in their communications. Some instant messaging applications 
offer the minimal assistance of converting or replacing a handful of widely used 
emoticon forming sequences of characters to corresponding graphical symbols. For 
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example, when the characters ":" (colon), "=" (equal sign) and ")" (right parenthesis) 
are successively entered, some instant messaging applications automatically 
replace the entered characters with the graphical symbol " ". 

These prior art facilities suffer from a number of disadvantages. First, 
5 typically only a handful of these emoticon forming character sequences are 

supported. Thus, virtually no assistance is provided to a user who chooses to be 
creative, and uses an unconventional sequence of characters to form an emoticon, 
e.g. the character sequence of "#" and "0" for a flabbergasted or astounded 
expression. Further, regardless whether the character sequence is conventional or 
Q 10 unconventional, a user typically has to enter the emoticon forming characters one at 

i fe 3 a time. This disadvantage is amplified in situations where the user is conducting the 

W 

fu textual or non-verbal communication using a communication device having limited 

input facilities, such as wireless mobile phones. 

!. v Accordingly, facilities that are more user friendly in assisting a user to employ 

?a 15 emoticons in their communications, especially on communication devices with 

H limited input facilities, such as wireless mobile phones, are desired. 
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BRIEF DESCRIPTION OF DRAWINGS 

The invention is illustrated by way of example and not by way of limitation in 
the figures of the accompanying drawings, in which like references indicate similar 
5 elements and in which: 

Figure 1 illustrates a wireless mobile phone incorporated with the teachings 
of the present invention for assisting a user in inputting an emoticon, in accordance 
with one embodiment; 

Figure 2 illustrates a state of the wireless mobile phone of FIG. 1 , where a list 
10 of emoticons is displayed for user selection, responsive to an initial selection of an 
=2 input key; 

W Figure 3 illustrates another state of the wireless mobile phone of FIG. 1 , with 

\M the current focus being placed on one of the displayed emoticons, as directed by the 

j ; y user traversal actions on the displayed list; 

15 Figure 4 illustrates yet another state of the wireless mobile phone of FIG. 1 , 

; ; ~ with a selected emoticon incorporated as part of a textual message being 

^ composed; 

\*a Figure 5 illustrates the relevant software elements of the wireless mobile 

phone of Fig. 1, for practicing the present invention, in accordance with one 
20 embodiment; 

Figure 6 illustrates the relevant hardware elements of the wireless mobile 
phone of Fig. 1, for practicing the present invention, in accordance with one 
embodiment; 

Figure 7 illustrates the operational flow of the relevant aspect of the emoticon 
25 input support logic provided in the key driver of Fig. 5, in accordance with one 
embodiment; 
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Figures 8a-8b illustrate alternate associations of the emoticon input logic to 
other input keys; 

Figure 9 illustrates yet another alternate association of the emoticon input 
logic to other input keys, employing an additional key; 

Figure 10 illustrates yet another alternate association of the emoticon input 
logic to other input keys, employing a removal cover or skin; and 

Figure 11 illustrates another mobile communication device incorporated with 
the teachings of the present invention, in accordance with another embodiment. 
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The present invention includes the provision of emoticon input logic to 
enhance the usability of a communication device. For ease of understanding, the 
present invention will be primarily described in the context of wireless mobile 
phones. However, the present invention is not so limited. The present invention 
may also be practiced on other communication devices besides wireless mobile 
phones. 

In the description to follow, various aspects of the present invention will be 
described. For purposes of explanation, specific numbers, materials and 
configurations are set forth in order to provide a thorough understanding of the 
present invention. However, in selected instances, well-known features are omitted or 
simplified in order not to obscure the present invention. The present invention may be 
practiced with only some of the aspects described, and without some of the specific 
details enumerated. 

Various operations will be described as multiple discrete steps in turn, in a 
manner that is most helpful in understanding the present invention. However, the 
order of description should not be construed as to imply that these operations are 
necessarily order dependent. In particular, these operations need not be performed in 
the order of presentation. 

Parts of the description will be presented in terms of operations performed by a 
processor based device, using terms such as displaying, focusing, selecting, and the 
like, consistent with the manner commonly employed by those skilled in the art to 
convey the substance of their work to others skilled in the art. As well understood by 
those skilled in the art, the quantities take the form of electrical, magnetic, or optical 
signals capable of being stored, transferred, combined, and otherwise manipulated 



Nelson - Emoticon Input Method 
And Apparatus 



5 



Express Mail Label No: 
EL910784090US 



Attorney Docket Ref: 41051 .P011 
CttentRef.:GW10.143 

through mechanical, electrical and optical components of the processor based device. 
The term processor as used in this specification includes microprocessors, micro- 
controllers, digital signal processors, and the like, that are standalone, adjunct or 
embedded. 

5 The description repeatedly uses the phrase "in one embodiment". Ordinarily 

the phrase does not refer to the same embodiment, although it may. The terms 
"comprising", "including", and "having" are frequently used in the specification and in 
the claims; the terms are synonymous. 

□ 10 Overview 

! -S 

! -3 Referring now to Figure 1, wherein a wireless mobile phone 100 incorporated 

i a y 

ft! with the teachings of the present invention, in accordance with one embodiment, is 
shown. As illustrated, for the embodiment, wireless mobile phone 100 comprises 

display 102 and input keys 104, including in particular, input key 110. Wireless mobile 

□ 

i:0 15 phone 100 operates in at least two modes, a voice or verbal mode, and a text or non- 
vi verbal mode. Further, wireless mobile phone 100 is endowed with emotion input logic 
|T (not shown) to facilitate a user in inputting emoticons, e.g. for a text message during 
the text or non-verbal mode of operation. For the embodiment, the emoticon input 
logic is associated with input key 110. That is, the emoticon input logic is activated by 
20 input key 110, e.g. while wireless mobile phone 100 operates in the text mode. As will 
be described in more detail below, using input key 110, supported by the emoticon 
input logic, a user may enter an emoticon e.g. into a text message in a more user 
friendly manner. 

Display 102 is provided to facilitate display of data, such as a textual message 
25 received or being composed, status information, such as "calling ..." or "low battery", 
choice of applications, such as "calendar" or "phone book", choice of options, such as 
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"AM" or "PM" (when setting a system clock), and so forth. The term "text" and its 
related forms, such as "textual", as used in the present application, are synonymous 
with the term "non verbal". In other words, a "textual" message may comprise of 
characters from a standard character set as well as "special" characters, symbols, 
5 glyphs, icons and the like. 

Input keys 104 are provided to facilitate input of data, command, user 
responses and/or selections to phone 100. For the embodiment, input keys 104 
include control keys 106 and alphanumeric keys 108; and as described earlier, the 
emoticon input logic is associated with input key 110 of alphanumeric keys 108. 
□ 10 Control keys 106 may include "make call" or "end call" function keys, as well as 

"scrolling" keys to facilitate a user in directing scrolling direction, i.e. up, down, to the 
right or to the left, when scrolling through an enumerated list of options or choices. 
Alphanumeric keys 108, for the embodiment, has a total of 12 alphanumeric keys, 
arranged in a 4x3 array configuration, that is 4 rows and 3 columns, with input key 
S 15 110 (associated with the emoticon input logic) disposed at the 4 th row and 1 st column 
position (with the row and column positions determined from top to bottom, and left to 
right respectively). Alphanumeric keys 108 are provided to facilitate input of 
alphanumeric data, including in particular, the input of emoticons through input key 
110 (while phone 100 operates in the text mode). 

20 

Operations 

Figures 2-4 illustrate a contemplated usage of input key 110 and the 
associated emoticon input logic (while phone 100 operates in the text mode). More 
specifically, Fig. 2-4 illustrate the external visible states of wireless mobile phone 100 
25 under the contemplated usage. As illustrated in Fig. 2, in response to an initial 

selection of input key 110 (while phone 100 is operating in the text mode, e.g. when a 
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textual message is being composed), a list of emoticons 112 is displayed on display 
102 for user selection. More specifically, for the illustrated embodiment, a list of 
emoticon forming character sequences is displayed on display 102 for user selection. 
For example, the "smiling face" emotion is displayed by displaying the three character 
5 sequence comprising the ":" (colon), the "-" (hyphen) and the ")" (right parenthesis) 
character. In alternate embodiments, singular graphical symbols, such as ©, may be 
employed instead. 

For the illustrated embodiment, emoticon forming character sequences 112 
(hereinafter simply emoticons) are displayed in a left-to-right, then top-to-bottom 

; : 

5 10 arrangement. In alternate embodiments, e.g. in support of other languages, such as 

h 

Hebrew, Chinese and so forth, emoticons 112 may be displayed in a right-to-left, then 

top-to-bottom arrangement, a top-to-bottom then left-to-right arrangement, or a top-to- 

iiy bottom then right-to-left arrangement. 

L For the illustrated embodiment, one of displayed emoticons 1 12 is also the 

15 "current focus" emoticon. Initially, under the left-to-right then top-to-bottom display 

^ arrangement, the leftmost emoticon among the emoticons displayed in the top row is 

□ 

k given the "current focus". In one embodiment, placing the "current focus" on a 

emoticon formed with a sequence of characters comprises highlighting all the 
characters. The term "highlighting" as used in the present application, including the 

20 claims, includes all known techniques to cause a sequence of characters to be 
visually distinct from other neighboring sequences of character, to draw a user's 
attention to the "highlighted" sequence of characters. These known techniques 
include but are not limited to underlying, italicizing or employing bold faces for the 
sequence of characters. 

25 As illustrated in Fig. 3, by selecting input key 110 again in a timely manner 

(thereby preventing an associated timer to expire), a user may scroll through the 
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displayed list of emoticons 112. The emoticon having the "current focus" is updated 
correspondingly as the user scrolls through the list. In alternate embodiments, other 
approaches, such as using one or more of control keys 106, may be practiced to 
facilitate scrolling through the displayed list of emoticons. In one embodiment, 
5 scrolling through emoticons formed with character sequences comprises scrolling 
through the character sequences, i.e. moving from one character sequence to 
another character sequence. 

As illustrated in Fig. 4, for the embodiment, upon elapse of a predetermined 
amount of time after the last selection of input key 110 (allowing the earlier 
10 mentioned associated timer to expire), the emoticon of the "current focus" is 
"2 selected, and inserted into e.g. text message 114 being composed. In alternate 

W embodiments, selection of the "current focus" emoticon may be made in other 

Lh manners, e.g. by "double clicking" input key 110 (i.e. successively clicking input key 

* ns 

I'D 110 at a predetermined "rapid" rate) or by selecting other input keys. Further, in 

g 15 alternate usages, it is contemplated that the selected emoticon may be used in a 
'{2 standalone manner instead, e.g. transmitted to a recipient directly (without having 

^ incorporated into a textual message). In one embodiment, selecting emoticon 

^ (formed with a sequence of characters) with the "current focus" comprises selecting 

all the characters of the emoticon with the "current focus". 
20 Note that upon selection, the characters of an emoticon forming character 

sequence (whether the sequence is being used in a standalone manner or as part of 
a text message) may be individually edited. For example, upon selection of the 
three character sequence ":", "-" and ")", one or more of the three characters ":", "-" 
and ")" may be individually edited, allowing a user to modify an emoticon to his/her 
25 liking. The "-" character may be modified to "#", or the ")" character may be modified 
to "0", and so forth. 
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Software and Hardware 
Figures 5-6 illustrate a software and a hardware component view of the 
relevant elements of wireless mobile phone 100, in accordance with one 
embodiment. As illustrated in Fig. 5, the relevant software elements 500 of phone 
5 100 includes textual communication component 502 provided to facilitate textual 
communication, display driver 504 provided to facilitate rendering of display onto 
display 102, and input driver 506 provided to support usage of input keys 104. Input 
driver 506 is incorporated with the earlier described emoticon input logic to support 
the earlier described emoticon input process. As illustrated, input driver 506 
10 provides textual communication component 502 with the inputs received through 
! £ ; input keys 104 (when textual communication component 502 is the "current" 

^ component having given control to receive the services of key driver 506). In the 
i*8 case of input key 110, while phone 100 is operating in the text mode, input driver 

i-i : 

iy 

ih 506 provides textual communication component 502 with the specific "selected" 

dj 15 emoticon, after having assisted the user in selecting one of the available emoticons. 

: sl Fig. 6 illustrates the relevant hardware components of wireless mobile phone 

Q 

; s jl 100 for practicing the present invention, in accordance with one embodiment. As 

illustrated, the relevant hardware components 600 include storage medium 604 and 
processor 602. For the embodiment, the relevant hardware components 600 also 

20 include general purpose input/output (GPIO) interface 606 and video controller 608. 
The elements are coupled to each other via bus 610, 

Storage medium 604 is provided to store the programming instructions 
implementing key driver 506. In one embodiment, storage medium 604 is a type of 
non-volatile memory, which is also employed to store the programming instructions 

25 implementing display driver 504 and textual communication component 502. 
Processor 602 coupled to storage medium 604 is provided to execute the 
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programming instructions implementing key driver 506. In one embodiment, 
processor 602 is also employed to execute the programming instructions 
implementing display driver 504 and textual communication component 502. GPIO 
606 and video controller 608 are provided for coupling input keys 104 and display 
5 102 respectively. 

Figure 7 illustrates the operational flow of the relevant aspects of key driver 
506 in support of the earlier described emoticon input process. As illustrated, upon 
detecting the selection of key 110 while phone 100 is operating in the text mode, key 
driver 506 causes a list of emoticons 112 to be displayed for user selection, block 
Q 10 702. Upon displaying the list of emoticons 112, key driver 506 further places a 

"current focus" on the "first" emoticon displayed, block 704. 
I S U Thereafter, key driver 506 determines if a predetermined associated timer has 

\Q expired, block 706. If the associated timer has not expired, key driver 506 
3 determines if another input key has been selected, block 707. If another input key 

m 15 has not been selected, key driver 506 determines if input key 110 is selected again, 
y block 708. The determinations of blocks 706-708 are repeated continuously, until 

Q 

L eventually either input key 110 is re-selected before another input key is selected or 
the associated timer expires, or another input key is selected, or the associated 
timer expires. 

20 If input key 110 is re-selected before another input key is selected or the 

associated timer expires, key driver 506 interprets the user action as a scrolling 
action, and updates the "current focus" to reflect the user's scrolling action 
correspondingly, block 710. Further, key driver 506 resets the associated timer. For 
the embodiment, scrolling beyond the end of the list causes the "current focus" to be 

25 placed back on the first emoticon. In other words, the user may scroll through the 
emoticons in a round-robin fashion, and return to select a "passed" emoticon. In 
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alternate embodiments, where control keys 106 are employed to facilitate scrolling, 
naturally, scrolling in either forward or backward direction may also be supported. 

Eventually, either the user selects another input key, or after the user stops 
selecting input key 110 for the predetermined time period, the associated timer 
5 expires. At such time, key driver 506 sets the emoticon having the "current focus" as 
the selected emoticon, and notifies textual communication 502 accordingly (which in 
the earlier described contemplated usage results in the selected emoticon being 
incorporated as part of the textual message being composed), block 712. 

As described earlier, the emoticons may be character sequences of multiple 
3 10 characters or single pixel-map based graphical symbols. For emoticons 
uj implemented through character sequences, the individual characters of the selected 

j p character sequence become individually editable upon incorporation into a textual 
?f t message. A single pixel-map based graphical symbol, upon selection, may of 
L course nevertheless be deleted from the textual message. 
! s 5 15 Thus, it can be seen from the above description, usability of phone 100 is 

^ improved, with the advantageous provision of the emoticon input logic. 

Alternate Embodiments 
Figures 8a-8b illustrate a number of alternate embodiments for associating 
20 the emoticon input logic of the present invention with input keys. As illustrated in 
Fig. 8a, in lieu of being associated with the "lower left corner" input key (which is 
often the (asterisk) key when phone 100 js operating in the voice mode), emotion 
input logic may be associated with the "lower right corner" key (which is often the "#" 
(pound) key when phone 100 is operating in the voice mode) instead. Additionally, 
25 as illustrated in Fig. 8b, in lieu of being associated with either the "*" or the "#" key, 
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emotion input logic may be associated with an actual "number" key instead, e.g. the 
number "5" key as shown, or other number keys, e.g. the "0" key. 

Figure 9 illustrates yet another alternate embodiment for associating 
emoticon input logic with an input key. As illustrated, in lieu of being associated with 
5 one of the keys of the 12-key arrangement 108, emoticon input logic may be 
associated with a dedicated function key that is in addition to the 12-key 
arrangement instead. Dedicated function key 1 10 is provided exclusively for the 
purpose of facilitating the earlier described emoticon input process. 

Figure 10 illustrates yet another alternate embodiment for providing the 
j 5 10 emoticon input logic of the present invention, and associating the logic with an input 
! *5 key. As illustrated, in lieu of directly providing the emoticon input logic to phone 100, 

! U and associating the logic with one of the keys of the 12-key arrangement 108, 
S emoticon input logic and its association with one of the input key 1 10 is indirectly 
« implemented on a removable cover or skin 1000 having a storage medium 1010, 

Co 15 within which the enhanced key driver 506 is disposed. Additionally, storage medium 
%j 1010 may also include the input key association information. 

2 As illustrated, cover or skin 1000 also includes a number of cutouts 1002- 

1004to facilitate "mating" with wireless mobile phone 100. Naturally, the sizes and 
shapes, as well as the precise placements of the cutouts are target device 
20 dependent. For the embodiment, wireless mobile phone 100 is designed to be able 
to receive removable cover or skin 1000. Further, wireless mobile phone 110 is 
designed to be able to access storage medium 1010 for key driver 506 (and key 
association information if any). Thus, the usability of wireless mobile phone 100 
may be "field" enhanced by having removable cover 1000 attached to it in the "field" 
25 (i.e. post initial sale). 
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Note that different collections of emoticons, and/or different input key 
associations may be provided to different families of removable covers 1000. 

Alternatively, in lieu of disposing the implementing programming instructions 
and input key association information in storage medium 1010, some or all of the 
5 implementing programming instructions and association information may be 
replaced by a resource identifier (such as a URL) instead. The replacement 
resource identifier identifies a location where the replaced implementing 
programming instructions and input key association information may be retrieved. 
Removable or interchangeable cover endowed with the capabilities to 
10 personalize or enhance the functionalities of a communication device is the subject 
^ matter of provisional application number <to be inserted>, entitled "Personalizing 
| J Electronic Devices and Smart Covering", filed on July 17, 2001. The specification of 

~ the provisional application is hereby fully incorporated by reference. 

! y 

* Figure11 illustrates another communication device incorporated with the 

j °0 15 teachings of the present invention. For the illustrated embodiment, communication 
H device 1100 is a palm sized personal digital assistant equipped with email and/or 
M= instant messaging capability. Communication device 1100 is also equipped with 

display 1102, input key 1110 and the complementary key driver logic as earlier 
described for wireless mobile phone 100. Accordingly, it is readily apparent that the 
20 present invention may be practiced on a wide range of communication devices. 



Conclusion and Epilogue 
Thus, a novel emoticon input method and a number of apparatuses embodying 
the supporting logic for the method have been described. While the present invention 
25 has been described in terms of the above illustrated embodiments, those skilled in the 
art will recognize that the invention is not limited to the embodiments described. For 
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example, in selected embodiments, in lieu of presenting a plurality of emoticons for 
selection, the emoticons may be presented for selection one at a time, with each 
"single clicking" reselection of the input key causing another emoticon presented for 
selection and a "double clicking" reselection of the input key (or selection of another 
5 input key or timeout) denoting selection of the displayed emoticon. Further, in others 
or the same selected embodiments, the list of emoticons available for selection may 
be user editable. That is, additional facilities (similar to e.g. name entries for an 
address book application) may be provided for a user to add to or subtract from a list 
of default emoticons provided by a vendor. Accordingly, the present invention can be 

^ 10 practiced with modification and alteration within the spirit and scope of the appended 
claims. The description is thus to be regarded as illustrative instead of restrictive on 

; y the present invention. 
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